Some thoughts on the paper's analysis techniques.
Quote - "To remove background noise, 3D convolutions were performed using average and Gaussian filters of size 23 3 23 3 43 pixels (x,y,t) on time-series image stacks (I(x,y,t))"
Thoughts - I have already demonstrated in the past that using the Video Block Matching 3D denoising algorithm is superior to using Gaussian Filter denoising. In that regard we are still ahead of the game.
Quote - "Then a noise filtered image stack (Ifil(x,y,t)) was obtained by subtracting the 3D convolution products of average and Gaussian filtering. To determine the locations of regions exhibiting dynamic changes in fluorescence, the mean intensity (Ibg) and standard deviation (sbg) of background pixels on sum-intensity projected noise filtered stacks (Ifil(x,y,t)) was calculated."
Thoughts - This is an unnecessarily convoluted explanation. In addition, the following excerpt can mean many things subtracting the 3D convolution products of average and Gaussian filtering. This is too vauge, for me at least.
Quote - "Next, the sum-intensity projected image was binarized using a threshold value of Ibg + 2sbg. After smoothing the image with a 43x3x43 pixels Gaussian filter and standard deviation of 3 pixel, regional maximas were detected on the sumintensity projected image."*
Thoughts - So they find regional maximas after binarizing and smoothing the data? They really beat this data into submission with their preprocessing. I'm not saying it is wrong, but it is destroying much of the actual signal. This is classic segmentation preprocessing. It is also a notable benefit of using the techniques we're using. The probabilistic techniques do not need such biased preprocessing. We will merely remove the noise that we "know" is there and let the algorithm work with all of the data. This is something I write about as well.
Quote - "To generate a mask of putative microdomains, all microdomains with more than two regional maxima using a marker-controlled watershed segmentation method were segmented, and microdomains with an area less than 5x3x5 pixels were considered at noise level and removed from the binarized image."
Thoughts - They are indeed segmenting. But again, there is a large amount of bias and preconception going into this technique. I hope not to sound too enamored with our own ideas, but again using our techniques we have little bias. The regions in latency space that we are favoring take ALL of the statistical properties of the data into consideration. We're merely identifying regions of the natural data that have common statistical structure.
Quote - "Then based on this binarized mask of potential microdomains, the intensity level over time of individual microdomains from the raw time series image stack I(x,y,t)) was plotted. The intensity profile of each microdomain was normalized by subtracting the mean intensity value of its background and then to rescale, this value was divided by the standard deviation of the background. In this study, this value is represented as a modified z-score (z)."
Thoughts - Considering it as a modified z-score is a clever idea. I like that. That allows for general probabilistic analysis if desired. That is something we can use too if we introduce a bit more traditional analysis into our pipeline.
Quote - "Microdomain events were then automatically detected based on the following criteria: to consider an event to be positive it must consist of a regional maximum of more than 5.0 a.u. within 4 neighboring frames, the beginning and end time point of each event was identified based on the time point prior to or after the peak that had an intensity value larger than 0.5 a.u. Events that spanned less than 4 frames were excluded."
Thoughts - This is reminiscent of the smorgasbord of hand crafted features/criteria we were extracting from the flow along paths to detect hotspots in the past. That was honestly a cludgy and inelligent approach. As far as hotspot detecting is concerned, our new approach with VAES is more grounded theoretically and significantly easier (no hand drawing paths on every astrocyte, no labeling, etc)
Quote - "The Support Vector Machine (SVM) algorithm was used to detect active microdomains based on 75 parameters of each individual event."
Thoughts - SVMs are a great choice for this task. As far as two class classification problems are concerned in theory they are the best approach second only to Multilayer Perceptron (MLP) neural networks. Though in practice the two techniques perform roughly the same.
Quote - "These parameters were extracted from a raw and two smoothed intensity profiles with different degree of smoothness. The smoothed intensity profiles were obtained from the difference between convolving raw intensity profiles with a Gaussian filter with unit standard deviation and an averaging filter of the same size. Two smoothed intensity profiles were obtained using filters of size 11 and 21 frames separately. Twenty-five parameters each were used to describe each individual event with these three intensity profiles. For initial training, the SVM algorithm was provided a set of 2500 fluorescence signals manually categorized as positive or negative, and various parameters of these events were normalized to a range between 0 and 1."
Thoughts - Oh my goodness... This is the point I was trying to get across to Monica back when we were working with the paths. We only had around 9 parameters and she wanted to manually choose their optimal settings for classifiction. That is, for all practical purposes, an impossible task. They had to decide on 25 features (which they did not specify) and then calculate them for 2500 manually labeled events... This is the most dramatic benefit in using our approach. In theory, once it is implenented, I can go from your raw dataset to labeled and extracted hotspots in just a few minutes with a single pass through a trained VAE (though training it may take several minutes, maybe even an hour, depending on the architecture we choose).
Quote - "These normalized parameters were used to generate a SVM model using libsvm. This SVM prediction model was applied to all events, and events classified as negative by libsvm were excluded. The overall accuracy from SVM prediction was > 85%, assessed by comparing the predicted results from an additional set of manually registered 2500 events, and 95% based on fluctuations in Ca2+-insensitive fluorescent protein tdTomato (Figures S1K–S1P)."
Thoughts - Oh those poor researchers. They actually had to hand label 5000 events! Accuracy of 85% on their test set, is not spectacular. That means for the events they KNOW to be true they could only identify 85% of them. They're missing some real events and including some false positives. Of course we will have to deal with that as well. I honestly don't know what that last sentence means about the 95%...
In [ ]: